There are several current approaches for removing visual objects based on the replacement of regions such as patches or fragments.
Wexler, Y.; Shechtman, E. and Irani, M.: “Space-Time Video Completion” in IEEE Trans. Pattern Anal. Mach. Intell. 29, 3 Mar. 2007, pages 463-476 presented a method for space-time completion of large space-time “holes” in video sequences of complex dynamic scenes. That method is based on an approach which applies a global optimization strategy. An objective function provides the means for measuring the coherence between two images. Rather small patches extending over space and time of an image sequence are used as a basis. That approach results in a highly sophisticated completion and synthesis of damaged image regions as well as missing frames.
Barnes, C.; Shechtman, E.; Finkelstein, A. and Goldman, D.: “PatchMatch: A Randomized Correspondence Algorithm for Structural Image Editing” in ACM SIGGRAPH 2009 Papers, H. Hoppe, Ed. SIGGRAPH '09, ACM, New York, pages 1-11 presented interactive image editing tools using a randomized algorithm
Those tools allow image completion as well as retargeting and reshuffling of images. For this, a randomized nearest neighbor algorithm is applied. This significantly speeds up the overall image generation even allowing for interactive image manipulations for rather small images.
A computer-implemented method for completing images is presented in US 2009/0003702 A1. That method receives images having holes, identifies the holes, and searches an image database to locate images that fill the holes.
WO 2010/129705 A1 discloses a computer-implemented method for generating a modified image. An image is displayed in a computer system. That image includes contents that have a feature visible therein. The contents have a region thereof defined to be provided with additional content in generating a modified image. An input is received that includes a mark indicating a region. Additional content for the region is determined using a patch-based optimization algorithm. The modified image is stored having the additional content in the region. The region may include a hole in the image without the contents wherein the modified image is generated in a hole-filling process.
A method for measuring bi-directional similarity between two signals is shown in WO 2008/139465 A2. According to that method, at least some patches of the first signal are matched with some patches of the second signal. Further, at least some patches of the second signal are matched with some patches of the first signal. A bi-directional similarity measure is calculated as a function of the matched patches.